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Abstract: This paper concerns estimating a probability density function / 
based on iid observations from g{x) = w{x) f{x), where the weight func- 

tion w and the total weig ht W = J w{x)f{x)dx may not be known. The 
length-biased and excess life distribution models are considered. The asymp- 
totic normality and the rate of convergence in mean squared error (MSE) of 
the estimators are studied. 



1. Introduction and preliminaries 

It is known from the famous "moment problem" that under suitable conditions a 
probability distribution can be recovered from its moments. In Mnatsakanov and 
Ruymgaart [5, 6] an attempt has been made to exploit this idea and estimate a cdf 
or pdf, concentrated on the positive half-line, from its empirical moments. 

The ensuing density estimators turned out to be of kernel type with a convolu- 
tion kernel, provided that convolution is considered on the positive half-line with 
multiplication as a group operation (rather than addition on the entire real line). 
This does not seem to be unnatural when densities on the positive half-line are to 
be estimated; the present estimators have been shown to behave better in the right 
hand tail (at the level of constants) than the traditional estimator (Mnatsakanov 
and Ruymgaart [6]). 

Apart from being an alternative to the usual density estimation techniques, the 
approach is particularly interesting in certain inverse problems, where the moments 
of the density of interest are related to those of the actually sampled density in a 
simple explicit manner. This occurs, for instance, in biased sampling models. In such 
models the pdf / (or cdf F) of a positive random variable X is of actual interest, 
but one observes a random sample Yi , . . . , Y„ of n copies of a random variable Y 
with density 

(1-1) 9iy)^^ ^{y)f{y), y>o, 

where the weight function w and the total weight 

poo 

(1.2) W= w{x)f{x)dx, 
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may not be known. In this model one clearly has the relation 

(1.3) /Ufe,F= / x^f{x)dx = W —f-^ g{y)dy, k = Q,l,..., 

Jo Jo w{y) 

and unbiased -v/n-consistent estimators of the moments of F are given by 

If w and W are unknown they have to be replaced by estimators to yield /tj,, say. In 
Mnatsakanov and Ruymgaart [7] moment-type estimators for the cdf F of X were 
constructed in biased models. In this paper we want to focus on estimating the den- 
sity / and related quantities. Following the construction pattern in Mnatsakanov 
and Ruymgaart [6] , substitution of the empirical moments Jik in the inversion for- 
mula for the density yields the estimators 

1.5 Ux) = — ^ — — ■ — — ■(-Yi) exp— , a; > 0, 

n wiYi) .T-la— 1)! \x J x 

i—l ' ' 

after some algebraic manipulation, where a is positive integer with a = a{n) ^oo, 
as n— > DO, at a rate to be specified later. liW ovw are to be estimated, the empirical 

moments /x^. are substituted and we arrive at say. 

A special instance of model (1.1) to which this paper is devoted for the most 
part is length-biased sampling, where 

(1.6) w{y) =y, y>0. 

Bias and MSE for the estimator (1.5) in this particular case are considered in 
Section 3 and its asymptotic normality in Section 4. Although the weight function 
w is known, its mean W still remains to be estimated in most cases, and an estimator 
of W is also briefly discussed. The literature on length-biased sampling is rather 
extensive; sec, for instance Vardi [9], Bhattacharyya et al. [1] and Jones [4]. 

Another special case of (1.1) occurs in the study of the distribution of the excess 
of a renewal process; see, for instance, Ross [8] for a brief introduction. In this 
situation, it turns out that the sampled density satisfies (1.1) with 

(1-7) «,(y) = i^^ = .J_, y>0, 

where /li? is the hazard rate of F. Although apparently w and hence W are not 
known here, they depend exclusively on /. In Section 5 we will briefly discuss some 
estimators for /, hp and W and in particular show that they arc all related to 
estimators of g and its derivative. Estimating this 5 is a "direct" problem and can 
formally be considered as a special case of (1.1) with w{y) = 1, y > and W = 1. 
Investigating rates of convergence of the corresponding estimators is beyond the 
scope of this paper. Finally, in Section 6 we will compare the mean squared errors 
of the moment-density estimator /* introduced in the Section 2 and the kernel- 
density estimator fh studied by Jones [4] for the length-biased model. Throughout 
the paper let us denote by G(a, b) a gamma distribution with shape and scale 
parameters a and 6, respectively. We carried out simulations for length-biased model 
(1.1) with g as the gamma G(2, 1/2) density and constructed corresponding graphs 
for /* and fh- Also we compare the performance of the moment- type and kernel- 
type estimators for the model with excess life-time distribution when the target 
distribution F is gamma G(2, 2). 
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2. Construction of moment-density estimators and assumptions 

Let us consider the general weighted model (1.1) and assume that the weight func- 
tion w is known. The estimated total weight W can be defined as follows: 

1 " 1 ~i 
Substitution of the empirical moments 

in the inversion formula for the density (see, Mnatsakanov and Ruymgaart [6]) 
yields the construction 

(2.1) /4:,) = ^^ Tm-(-^0 eM'-Yi)- 

n ~^ w{Yi) x-(a—l)\\x/ x 

Here a is positive integer and will be specified later. Note that the estimator is 
the probability density itself. Note also that 

W = W + 0p{^), n^oo, 

(see, Cox [2] or Vardi [9]). Hence one can replace W in (2.1) by W. 

Investigating the length-biased model, modify the estimator fa and consider 

n X ■ (a - 1)1 \x J x 

i—l 

= -E^-f(^-(-j •exp(--i;) = -EM,. 

In Sections 3 and 4 we will assume that the density / satisfies 

(2.2) /e C(2)([0,oo)), with sup |/"(t)|= M < oo. 

t>o 

In Section 5 we will consider the estimator of the unknown survival function S given 
the sample li, . . . ,1^ from pdf (1.1) where w = {1 — F)/ f. Namely, we will use 
the moment-density estimator proposed in Mnatsakanov and Ruymgaart [6] which 
yields the estimator of S = 1 — F: 

^«(-) = -E^T^-(-^0"exp(-^^.). 

n ^ Yi {a -\)\ \x I X 

We will assume that F satisfies the conditions 

(2.3) FeC^^HlO.oo)). with sup |F"(t)|= i < oo. 

t>o 

Throughout the paper the moment-type estimators will be considered at a fixed 
point a; > 0, where f{x) > 0. 



Moment-density estimation 



325 



3. The bias and MSE of /* 

To study the asymptotic properties of /* let us introduce for each k G N the 
sequence of gamma G{k{a — 2) + 2, x/ka) density fimctions 

h ^(y\ - - ( — 1 u'=(«-2)+l 

.3 1. "'^-'^ ^ " {k{a-2) + lv\x) 

X expl u), u > 0, 

X 

with mean {k{a — 2) + 2]x/{ka) and variance {^(q: — 2) + 2}x^/(fca)2. For each 
fc e N, moreover, these densities form as well a delta sequence. Namely, 

POO 

/ ha,x,k (u) f{u) du f{x) , as a ^ oo , 
Jo 

uniformly on any bounded interval (see, for example. Feller [3], vol. II, Chapter 
VII). This property of ha,x,k, when fc = 2 is used in (3.10) below. In addition, for 
fc = 1 we have 



(3.2) / u ha,x,i {u) du = X, 



f°° x^ 

(3.3) / {u — xY ha X 1 {u) du = — . 

Theorem 3.1. Under the assumptions (2.2) the bias of f* satisfies 

(3.4) Ef*{x) - fix) = ^^f^ +o(-), as a ^oo. 

2 ■ a \a J 



For the Mean Squared Error (MSE) we have 



(3.5) MSE{f*a{x)} = n 



= r,-4/5 



w-f{x) , x\nx)} 



20Fa;2 

provided that we choose a = a{n) ^ rP'l'° . 
Proof Let Mi = W ■ Y'^ ■ ha,x,i [Yi). Then 



+ 



+ 0(1), 



EM^ = W''-Yr^hl^,^^{y)g{y)dy 
Jo 

(3.6) 



{y{a-l)\}'^ KxJ " ^ J 

In particular, for A; = 1: 

(3.7) ^^"^'^ ^ ^«jj)=^r?-fR-&)"^^p(-^^^i^'^ 

= / h^^,^,{y)fiy)dy = EMi. 

Jo 
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This yields for the bias = x,a^ = /a) 



(3.8) 



fa{x)-f{x) = I ha,x,liy){f(y) - fix)}du 

Jo 

POO 

= / ha^a:,l{y){f{x) + iy-x)f{x 

Jo 

2/ 



1 r°° 

+ 7; (y- xfiriy) - .f{x)}dy 



1 f 

2 / {y - xfha,x,i{y)f"{x)du 

+ {y- xfh„,x,Mf"{y) - f"{x)}dy 

lx\ fl\ 

-— / {x)+o(-\, asa^cxD. 

2 a \a J 



For the variance we have 

(3.9) Var/:(x) = ^VarM, = ^{E Mf - /^(x)}. 

Applying (3.6) for A; = 2 yields 

^2 ^r g-(2n-:n|(2n,- 3)}'a"-^i) + V2 ^ 

^^V2^ e-2(«-i){(a-l)}2("-i)+i 22("-i) 
-,00 

X 



/ ha,x,2{u)f{u)du= I K,x,2{u)f{u)du 

Jo W'^ x^ Jo 

W x/a W x/a ^ 

^'^{f{x)^o{X)] = ^^^{x) + o{M 



2^ a;2 ^ ^ ^ 2^ x^ 
as a — > 00. Now inserting this in (3.9) we obtain 

„ „ , ^"Z-'^' = \ [2^ ^ ^<^* + - {«^' + ° (^) }' 

2n^ x^ \ n 
Finally, this leads to the MSE of 7*(a;): 

(3.1.) M..,/;(.„ = ^ M , . „ ) . „ 

For optimal rate we may take 

(3.13) a = Q!n ~ 

assuming that n is such that a„ is an integer. By substitution (3.13) in (3.12) wc 
find (3.5). □ 
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Corollary 3.1. Assume that the parameter a = a{x) is chosen locally for each 
X > as follows 



(3.14) a{x)=r^y^.{^Vf^ 



4/5 

■ fix) 



Then the estimator /^(a;) = /^(a;) satisfies 



f"{x) ^ 0. 



(3.16) MSE{f*„{x)} 



n 



-4/5 



w^-f"{x)-f^{xy'^' 



n ■ X- 



V2 



+ o(l), as n ^ oo. 



Proof. Assuming the first two terms in the right hand side of (3.12) are equal to each 
other one obtains that for each n the function a = a{x) can be chosen according 
to (3.14). This yields the proof of Corollary 1. □ 

4. The asymptotic normality of /* 

Now let us derive the limiting distributions of /* . The following statement is valid. 

Theorem 4.1. Under the assumptions (2.2) and a = a{n) ~ , for any < 6 < 2, 
we have, as a ^ oo, 

(4.1) Normal{0,l). 



\J Varf*{x) 



Proof. Let < C < oo denote a generic constant that does not depend on n 
but whose value may vary from line to line. Note that for arbitrary k G N the 
"cr-inequality" entails that E\M, - faix)\''< CEAff , in view of (3.6) and (3.7). 
Now let us choose the integer k > 2. Then it follows from (3.6) and (3.11) that 

.,2. ELiE|^{M,-/.(x)}|^ n^-^fc-V^g'^/^-V^ 
^ ■ ^ {Var/„(a;)}fc/2 " (n-iai/2)fe/2 

for a ^ . Thus the Lyapunov's condition for the central limit theorem is fulfilled 
and (4.1) follows for any < ^ < 2. □ 

Theorem 4.2. Under the assumptions (2.2) we have 

n^/^ _ / W ■ f(x) 

(4-3) ^ {/: W - m} Normal (o, 

as n ^ oo, provided that we take a = a{n) ~ for any | < (5 < 2. 

Proof. This is immediate from (3.11) and (4.1), since combined with (3.8) entails 
that ni/2a-V4{/^(a;) _ f(x)} = 0{n}'^ a-^'^) = 0{n-^) = o(l), as n ^ oo, 
for the present choice of a. □ 
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Corollary 4.1. Let us assume that (2.2) is valid. Consider /^(x) defined in (3.15) 
with a{x) given by (3.14). Then 

as n ^ 00 and f"{x) ^ 0. 

Proof. Prom (4.1) and (3.11) with a = a{x) defined in (3.13) it is easy to see that 

(4.5) £^ - EJH.)) = No™.l ( 0, ) + oK;^). 

as n ^ oo. Application of (3.4) where a = a{x) is defined by (3.14) yields (4.4). □ 

Corollary 4.2. Let us assume that (2.2) is valid. Consider /*. (x) defined in (3.15) 
with a*{x) given by 

(4.6) a*{x)=n'.{^y 

Then when f"{x) ^ 0, and letting n — » oo, it follows that 

„ ^ W fix) \ 

(4.7) {f:, (X) - fix)} TVorma/ (^0, j . 

Proof. Again from (4.1) and (3.11) with a = a*(a;) defined in (4.6) it is easy to see 
that 

(4-8) ^^{/^(x)-i?/^(^)}= Normal (o, +,^(i), 

as n — > oo. On the other hand application of (3.4) where a = a*ix) is defined by 
(4.6) yields 

(4-9) {^/:«(-) - /(-)} - O ( , 

as n ^ oo. Here C(a;) = {||iM}i/2_ Combining (4.8) and (4.9) yields (4.7). □ 



f"{x) 



- <S<2. 
5 



5. An application to the excess life distribution 



Assume that the random variable X has cdf F and pdf / defined on [0, oo) with 

F(0) = 0. Denote the hazard rate fimction hp = f /S, where S = 1 — F is the 
corresponding survival function of X. Assume also that the sampled density g 
satisties (1.1) and (1.7). It follows that 

(5-1) 9{y) = ^{i-F{y)}, 2/>o. 

It is also immediate that W = 1/5(0) and, fiy) = —Wg'iy) 
that 

My) = -^, y>o. 
9{y) 



= -^, y>o,so 
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Suppose now that we are given n independent copies Yi , . . . , 1^ of a random variable 
Y with cdf G and density g from (5.1). 

To recover F oi S from the sample Yi,. . . ,Yn use the moment-density estimator 
from Mnatsakanov and Ruymgaart [6] , namely 

i=l 

Where the estimator W can be defined as follows: 

1 



W 



m 

Here g is any estimator of g based on the sample Yi,. . . ,Yn. 

Remark 5.1. As has been noted at the end of Section 1, estimating g from 
Yi,...,Yn is a "direct" problem and an estimator of g can be constructed from 
(1.5) with W and w{Yi) both replaced by 1. This yields 

n y {a - ly. \y J y 

The relations above suggest the estimators 

Here let us assume for simplicity that W is known and construct the estimator 
of survival function S as follows: 

i=l ^ ' i=l 

Theorem 5.1. Under the assumptions (2.3) the bias of Sa satisfies 

(5.5) ESaix) - S(x) = -^JIM. +o(-\, as a ^ 00. 

2 ■ a \ a J 



For the Mean Squared Error (MSE) we have 



(5.6) MSE{Sa{x)} = n 



-4/5 



w ■ s{x) x^{f'{x)y 



2-x^/n 

provided that we choose a = a{n) ~ n^/^. 



+ 



o{l), 



Proof. By a similar argument to the one used in (3.8) and (3.10) it can be shown 
that 

POO 

(5.7) ESocix) - S{x) = ha,a:,i{u){S{u) - S{x)}du 

Jo 

= ---f{x)+o^-j,asa^oo 
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and 
(5.8) 



W 



-S{x) + o{\/a), asa ^ oo , 



2y/TT X 

respectively. So that combining (5.7) and (5.8) yields (5.6). 



□ 



Corollary 5.1. // the parameter a = a{x) is chosen locally for each x > as 
follows 



(5.9) a(x)=„V5.{_^}i/5.^2. 



fix) 



then the estimator (5.4) with a = a{x) satisfies 



4/5 



, fix) + 0. 



MSE{So,{x)] = n-^l^ 



■ fix) ■ (1 - F{x)f 



TrV2 



2/5 



+ o(l), as n — > oo. 



Theorem 5.2. Under the assumptions (2.3) and a = a{n) ~ for any < 6 <2 

we have, as n oo, 



(5.10) 



S„(x) - ESc{x) 



*d Normal {0, 1) . 



^JvarSc,{x) 

Theorem 5.3. Under the assumptions (2.3) we have 



(5.11) 



,1/2 



{S„{x) - S{x)} 



Normal 0, )J- , 



as n —> oo, provided that we take a = a{n) ^ for any | < 5 < 2. 

Corollary 5.2. If the parameter a = a{x) is chosen locally for each x > according 
to (5.9) then for Sa{x) defined in (5.4) we have 



,1/2 



^ {S^ix) - Six)} Normal 



WS{x).^,^ WS{x) 



\ 2x\/t: ' 2x\pK ) ' 



a(a;)i/4 

provided f'{x) ^ and n — > oo. 

Corollary 5.3. // the parameter a = a*{x) is chosen locally for each x > 
according to 



(5.12) «*(:,) =n^.{__}i/5. ^2. 



then for Sa* (x) defined in (5.4) we have 



fix) 



VI - F{x) 



4/5 



,-<^<2, 



,1/2 



^ {S^, (x) - S{x)} Normal 



WSjx) 



a*(a;)i/4 
provided f'{x) ^ and n — > oo. 

Note that the proofs of all statements from Theorems 5.2 and 5.3 are similar to 
the ones from Theorems 4.1 and 4.2, respectively. 
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6. Simulations 

At first let us compare the graphs of our estimator /* and the kernel-density esti- 
mator fh proposed by Jones [4] in the length-biased model: 



Assume, for example, that the kernel K{x) is a standard normal density, while the 
bandwidth h = 0{n~^), with < /3 < 1/4. Here W is defined as follows 



In Jones [4] under the assumption that / has two continuous derivatives, it was 
shown that as n — > oo 



Comparing (6.2) with (3.12), where a = h ^, one can see that the variance term 
Var/*(a;) for the moment-density estimator could be smaller for large values of x 
than the corresponding Ya.r{fh{x)} for the kernel-density estimator. Near the origin 
the variability of fh could be smaller than that of /*. The bias term of /* contains 
the extra factor x^, but as the simulations suggest this difference is compensated 
by the small variability of the moment-density estimator. 

We simulated n ~ 300 copies of length-biased r.v.'s from gamma G(2, 1/2). The 
corresponding curves for / (solid line) and its estimators /* (dashed line), and fh 
(dotted line), respectively are plotted in Figure 1. Here we chose a = r^l^ and 



(6.1) 





{%.2)MSE{fh{x)} = VarA(x) +bias2{M(a;) 
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Figure 1. 



Fig 1. 
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Fig 2. 

— n~^/^, respectively. To construct the graphs for the moment-type estimator 
Sa defined by (5.4) and the kernel-type estimator Sh defined in a similar way as 
the one given by (6.1) let us generate n — 400 copies of r.v.'s Yi, . . . ,Yn with pdf g 
from (5.1) with W — A and 

1 - F{x) = + I e^^ ,a; > 0. 

We generated Yi, . . . , 1^ as a mixture of two gamma G{\, 2) and G(2, 2) distribu- 
tions with equal proportions. In the Figure 2 the solid line represents the graph of 
S = 1 — F while the dashed and dotted lines correspond to Sa and Sh-, respectively. 
Here again we have a = r?l^ and h = n^^l^ . 
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